Pruning and Exclusion Criteria for Unordered Incremental Reduced Error Pruning

Incremental reduced error pruning is a technique that has been extensively used for efficient induction of ordered rule sets (decision lists). Several criteria have been developed regarding how to prune rules and whether or not to exclude generated rules. A version of incremental reduced error pruning for unordered rule sets is presented, and the appropriateness of previously proposed criteria for the novel version is investigated. It is shown that when inducing unordered rule sets, where a Bayesian framework is used to combine predictions from multiple rules, previously proposed criteria could lead to exclusion of possibly beneficial rules as well as to inclusion of harmful rules. Two alternative criteria are introduced, one based on the likelihood ratio and one based on the margin. An empirical evaluation on 34 datasets shows that the novel criteria significantly outperform previously employed criteria when using incremental reduced error pruning for unordered rule sets, the margin-based being slightly ahead of the likelihood ratio criterion.